Weakly supervised semantic segmentation (WSSS) with image-level labels is a challenging task in computer vision. Mainstream approaches follow a multi-stage framework and suffer from high training costs. In this paper, we explore the potential of Contrastive Language-Image Pre-training models (CLIP) to localize different categories with only image-level labels and without any further training. To efficiently generate high-quality segmentation masks from CLIP, we propose a novel framework called CLIP-ES for WSSS. Our framework improves all three stages of WSSS with special designs for CLIP: 1) We introduce the softmax function into GradCAM and exploit the zero-shot ability of CLIP to suppress the confusion caused by non-target classes and backgrounds. Meanwhile, to take full advantage of CLIP, we re-explore text inputs under the WSSS setting and customize two text-driven strategies: sharpness-based prompt selection and synonym fusion. 2) To simplify the stage of CAM refinement, we propose a real-time class-aware attention-based affinity (CAA) module based on the inherent multi-head self-attention (MHSA) in CLIP-ViTs. 3) When training the final segmentation model with the masks generated by CLIP, we introduced a confidence-guided loss (CGL) to mitigate noise and focus on confident regions. Our proposed framework dramatically reduces the cost of training for WSSS and shows the capability of localizing objects in CLIP. Our CLIP-ES achieves SOTA performance on Pascal VOC 2012 and MS COCO 2014 while only taking 10% time of previous methods for the pseudo mask generation. Code is available at https://github.com/linyq2117/CLIP-ES.

Previous work on action representation learning focused on global representations for short video clips. In contrast, many practical applications, such as video alignment, strongly demand learning the intensive representation of long videos. In this paper, we introduce a new framework of contrastive action representation learning (CARL) to learn frame-wise action representation in a self-supervised or weakly-supervised manner, especially for long videos. Specifically, we introduce a simple but effective video encoder that considers both spatial and temporal context by combining convolution and transformer. Inspired by the recent massive progress in self-supervised learning, we propose a new sequence contrast loss (SCL) applied to two related views obtained by expanding a series of spatio-temporal data in two versions. One is the self-supervised version that optimizes embedding space by minimizing KL-divergence between sequence similarity of two augmented views and prior Gaussian distribution of timestamp distance. The other is the weakly-supervised version that builds more sample pairs among videos using video-level labels by dynamic time wrapping (DTW). Experiments on FineGym, PennAction, and Pouring datasets show that our method outperforms previous state-of-the-art by a large margin for downstream fine-grained action classification and even faster inference. Surprisingly, although without training on paired videos like in previous works, our self-supervised version also shows outstanding performance in video alignment and fine-grained frame retrieval tasks.

Despite the tremendous progress of Masked Autoencoders (MAE) in developing vision tasks such as image and video, exploring MAE in large-scale 3D point clouds remains challenging due to the inherent irregularity. In contrast to previous 3D MAE frameworks, which either design a complex decoder to infer masked information from maintained regions or adopt sophisticated masking strategies, we instead propose a much simpler paradigm. The core idea is to apply a \textbf{G}enerative \textbf{D}ecoder for MAE (GD-MAE) to automatically merges the surrounding context to restore the masked geometric knowledge in a hierarchical fusion manner. In doing so, our approach is free from introducing the heuristic design of decoders and enjoys the flexibility of exploring various masking strategies. The corresponding part costs less than \textbf{12\%} latency compared with conventional methods, while achieving better performance. We demonstrate the efficacy of the proposed method on several large-scale benchmarks: Waymo, KITTI, and ONCE. Consistent improvement on downstream detection tasks illustrates strong robustness and generalization capability. Not only our method reveals state-of-the-art results, but remarkably, we achieve comparable accuracy even with \textbf{20\%} of the labeled data on the Waymo dataset. The code will be released at \url{https://github.com/Nightmare-n/GD-MAE}.

Existing neural rendering methods for creating human avatars typically either require dense input signals such as video or multi-view images, or leverage a learned prior from large-scale specific 3D human datasets such that reconstruction can be performed with sparse-view inputs. Most of these methods fail to achieve realistic reconstruction when only a single image is available. To enable the data-efficient creation of realistic animatable 3D humans, we propose ELICIT, a novel method for learning human-specific neural radiance fields from a single image. Inspired by the fact that humans can easily reconstruct the body geometry and infer the full-body clothing from a single image, we leverage two priors in ELICIT: 3D geometry prior and visual semantic prior. Specifically, ELICIT introduces the 3D body shape geometry prior from a skinned vertex-based template model (i.e., SMPL) and implements the visual clothing semantic prior with the CLIP-based pre-trained models. Both priors are used to jointly guide the optimization for creating plausible content in the invisible areas. In order to further improve visual details, we propose a segmentation-based sampling strategy that locally refines different parts of the avatar. Comprehensive evaluations on multiple popular benchmarks, including ZJU-MoCAP, Human3.6M, and DeepFashion, show that ELICIT has outperformed current state-of-the-art avatar creation methods when only a single image is available. Code will be public for reseach purpose at https://elicit3d.github.io .

我们介绍了Sparrow，这是一个寻求信息的对话代理，与提示的语言模型基线相比，训练有素，更有帮助，正确和无害。我们使用从人类反馈中的强化学习来培训我们的模型，以帮助人类评估者判断代理人的行为。首先，为了使我们的代理人更有帮助和无害，我们将良好对话的要求分解为代理人应遵循的自然语言规则，并分别向评估者询问每个规则。我们证明，这种崩溃使我们能够收集对代理行为的更多针对性的人类判断，并允许更有效的规则条件奖励模型。其次，我们的代理商在收集对模型声明的偏好判决时提供了支持事实主张的来源的证据。对于事实问题，麻雀提供的证据支持了78％的时间。比基线比基线更享受麻雀，同时对人类的对抗性探测更具弹性，在探测时只有8％的时间违反了我们的规则。最后，我们进行了广泛的分析，表明尽管我们的模型学会遵守我们的规则，但它可以表现出分布偏见。

尽管具有明显的区分靶向分布样本的能力，但深度神经网络在检测异常分布数据方面的性能差。为了解决此缺陷，最先进的解决方案选择在离群值的辅助数据集上训练深网。这些辅助离群值的各种培训标准是根据启发式直觉提出的。但是，我们发现这些直观设计的离群训练标准可能会损害分布学习，并最终导致劣等的表现。为此，我们确定了分布不兼容的三个原因：矛盾的梯度，错误的可能性和分布变化。基于我们的新理解，我们通过调整深层模型和损耗函数的顶级设计，提出一种新的分布检测方法。我们的方法通过减少对分布特征的概率特征的干扰来实现分布兼容性。在几个基准上，我们的方法不仅可以实现最新的分布检测性能，而且还提高了分布精度。

视觉变压器（VIT）是卷积神经网络（CNN）的强大替代方案，引起了很多关注。最近的工作表明，VIT也容易受到CNN等对抗性例子的影响。为了建立强大的VIT，一种直观的方法是应用对抗训练，因为它已被证明是完成强大CNN的最有效方法之一。但是，对抗性培训的一个主要局限性是其沉重的计算成本。 VIT所采用的自我注意力的机制是计算强度的操作，其费用随输入贴片的数量四次增加，从而使VIT上的对抗性训练更加耗时。在这项工作中，我们首先全面研究了有关各种视觉变压器的快速对抗训练，并说明了效率和鲁棒性之间的关系。然后，为了加快对VIT的对抗训练，我们提出了一种有效的注意力引导的对抗训练机制。具体而言，依靠自我注意的专长，我们在对抗训练过程中以注意引导策略的掉落策略积极地嵌入了每一层的某些斑块嵌入。纤细的自我发场模块大大加速了对VIT的对抗训练。只有65％的快速对抗训练时间，我们与具有挑战性的成像网基准相匹配。

仿真是用于创建控制策略和测试各种物理参数的机器人技术的重要步骤。 Soft Robotics是一个领域，由于可变形材料组件的非线性以及其他创新性且通常是复杂的物理特性而引起了独特的物理挑战，以模拟其主题。由于使用传统技术模拟柔软和异质物体的计算成本，刚性机器人模拟器不太适合模拟软机器人。因此，许多工程师必须构建自己为系统量身定制的一次性模拟器，或使用具有降低性能的现有模拟器。为了促进这项激动人心的技术的开发，这项工作为各种软机器人提供了交互式，准确和多功能的模拟器。我们的开源3D仿真引擎Cronos与可变形和刚性对象的超快速性能的质量弹簧模型平行。我们的方法适用于多种非线性材料构型，包括高变形性，体积致动或异质刚度。这种多功能性提供了在单个机器人模拟中自由混合材料和几何成分的能力。通过利用非线性胡克恩质量弹簧系统的灵活性和可扩展性，该框架通过高度并行模型模拟柔软而刚性的对象，以实现近实时速度。我们描述了有效的GPU CUDA实施，我们证明了该实施是为了在消费级GPU卡上实现每秒超过10亿个元素的计算。通过将结果与Euler-Bernoulli光束理论，固有频率预测和软结构在大变形下的软结构进行比较来验证系统的动态物理准确性。

实现通用语言情报是自然语言处理的长期目标，标准评估基准发挥基本和指导作用。我们认为，对于通用语言智能评估，基准本身需要全面和系统。为此，我们提出了Cuge，一种中文语言理解和生成评估基准，具有以下特征：（1）分层基准框架，其中数据集主要选择和组织语言能力 - 任务数据集层次结构。 （2）多级评分策略，其中基于分层框架提供了不同级别的模型性能。为了促进CUGE，我们提供了一个公共排行榜，可以自定义，以支持灵活的模型判断标准。代表性预先训练的语言模型的评估结果表明了对通用语言智能的完善的充足空间。 Cuge在Cuge.baai.ac.cn上公开提供。

本文提出了一种新的和富有激光激活方法，被称为FPLUS，其利用具有形式的极性标志的数学功率函数。它是通过常见的逆转操作来启发，同时赋予仿生学的直观含义。制剂在某些先前知识和预期特性的条件下理论上得出，然后通过使用典型的基准数据集通过一系列实验验证其可行性，其结果表明我们的方法在许多激活功能中拥有卓越的竞争力，以及兼容稳定性许多CNN架构。此外，我们将呈现给更广泛类型的功能延伸到称为PFPlus的函数，具有两个可以固定的或学习的参数，以便增加其表现力的容量，并且相同的测试结果验证了这种改进。